Machine Learning Basics

Most research approaches today are based on machine learning theory. A basic understanding of this topic is therefore required to understand the following pages.

A standard introduction to the field of machine learning is provided in the following book: R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, New York: John Wiley & Sons, 2001, ISBN: 0-471-05669-3.

The basic workflow of a machine learning approach is illustrated as follows:




Using training data, relevant features are extracted and passed to a machine learning algorithms. Examples are: Gaussian Mixture Models, Neural Networks, Support-vector Machines, Decision trees (for example ID3), k-Nearest Neighbors, Bayesian Networks, Hidden Markov Models. Feature extraction methods are usually derived from signal processing or electrical engineering (sensor input/output). Machine learning algorithms, are usually methods derived from mathematical statistics and produce statistical models. These are basically sparse representations of the data that allow thresholding of any kind. In order to train the models, the right answers have to be provided which are given in the ground truth data. This is usually metadata created by human annotators. In testing mode, the statistical models are then used to perform the actual pattern recognition task. The test data is of course run through the same feature extractionprocess. The results are either just used or compared against the ground truth in order to benchmark the quality of the algorithm. Supervised machine learning works just as described. Unsupervised machine learning omits the training step, statistical models are created on the fly using the test data.